Logic-Based Natural Language Understanding for Cognitive Tutors

High-precision Natural Language Understanding is needed in Geometry Tutoring to accurately determine the semantic content of students’ explanations. The paper presents an NLU system developed in the context of the Geometry Cognitive Tutor. The system combines unification-based syntactic processing with description logics based semantics to achieve the necessary accuracy level. The paper describes the compositional process of building the syntactic structure and the semantic interpretation of NLU explanations. It also discusses results of an evaluation of classification performance on data collected during a classroom study. 1 Explanations in Geometry Tutoring The Geometry Cognitive Tutor assists students in learning by doing as they work on geometry problems on the computer. Currently the Geometry Cognitive Tutor is in regular use (two days per week) in about 500 schools around the US. In previous evaluation studies Koedinger et al. (1997) have shown that the tutors are successful in raising high school students’ test scores in both algebra and geometry. However, there is still a considerable gap between the effectiveness of current cognitive tutor programs and the best human tutors (Bloom, 1984). Cognitive Tutors pose problems to students and check their solutions to these problems step by step. They can also provide context-sensitive hints at each step in solving the problem, as needed. However, prior Cognitive Tutors do not ask students to explain or justify their answers in their words. On the other hand human tutors often engage students in thinking about the reasons behind the solution steps. Such “selfexplanation” has the potential to improve students’ understanding of the domain, resulting in knowledge that generalizes better to new situations. This difference might also be the main explanation beneath the gap mentioned above. To verify this hypothesis, the next generation of intelligent cognitive tutors needs to be able to carry tutoring dialogs with students at the explanation level. Some of the current intelligent tutoring systems, like Autotutor (Wiemer-Hastings et al, 1999), Circsim-Tutor (Glass, 2000), and Atlas/Andes (Rosé et al, 2001), do have natural language processing capabilities. However, these systems rely on either statistical processing of language, identifying keywords in language, or some level of syntactic analysis. None of these approaches seem to achieve the degree of precision in understanding needed in a highly formalized domain such as geometry tutoring. 2 Logic-based Natural Language Understanding One of the main problems that the Geometry Tutor faces is to determine with accuracy the semantic content of students’ utterances. Natural language allows for many different ways to express the same meaning, all of which have to be recognized by the system as being semantically equivalent. The determination of semantic equivalence has to work reliably over variation of syntactic structure, variation of content words, or a combination of both. For example, the sentences below all express the same geometry theorem, about the measures of angles formed by other angles (the Angle Addition Theorem). An angle formed by adjacent angles is equal to the sum of these angles. The measures of two adjacent angles sum up to the measure of the angle that the 2 angles form. An angle's measure is equal to the sum of the two adjacent angles that compose it. The sum of two adjacent angles equals the larger angle the two are forming. The sum of the measures of two adjacent angles is equal to the measure of the angle formed by the two angles. The measure of an angle made up of two adjacent angles is equal to the sum of the two angles. If adjacent angles form an angle, its measure is their sum. When an angle is formed by adjacent angles, its measure is equal to the sum of those angles. An angle is equal to the sum of its adjacent parts. Two adjacent angles, when added together, will be equal to the whole angle. The sum of the measures of adjacent angles equals the measure of the angle formed by them. Two adjacent angles added together make the measure of the larger angle. The process also has to be consistent, so no unwarranted conclusions are derived from the text, and robust, in an environment of imprecise or ungrammatical language, as uttered more often than not by high school students. Many times this content equivalence relies on inferences specific to the domain of discourse. Our hypothesis is that such a high-precision recognition process needs to be based on contextual information about the domain of discourse modeled in a logic system. The paper presents an NLU system we have built to test this hypothesis. The next section describes the overall architecture of the system, and illustrates the main interpretation mechanism. Section 3 discusses the results of an evaluation we completed based on data from a recent classroom study. 2 The System’s Architecture The system’s overall architecture is presented in Figure 1 below. The interface module takes the input sentence from the tutor, word by word, in real time, and after some preprocessing and spelling correction, it passes it to a chart parser. The chart parser is the main engine of the system. It uses linguistic knowledge about the target natural language from the unification grammar and the lexicon. The parser used currently is LCFlex, a left-corner active-chart parser developed at Carnegie Mellon University and University of Pittsburgh (Rosé and Lavie, 2001). The parser takes words of a sentence one by one and combines them in larger phrase structures, according to